Mixture of normals with strong separation

Setup

The general form of the model is:

\[ \begin{align} y_{gi} &\sim N(\mu_{gi}, \sigma_g^2), \\ \mu_{gi} &= \beta_{g0} + x_i(\beta_{g1}D_g), \end{align} \] for \(g = 1, \dots, G\) and \(i = 1, \dots, N_g\).

In all cases, true values for the regression parameters are drawn from: \[ \begin{align} \beta_{g0} &\sim N(0, 1) ~\text{and}~\\ \beta_{g1} &\sim 0.5 N(-M, 1) + 0.5 N(M, 1). \end{align} \] To visualize the simulating distribution for \(\beta_{g1}\):

\(G = 50, N_g = 10\) and \(x_i = 0, i \leq 5\) and \(1\) otherwise.

  • Model 1: \(M = 10\), \(D_g = 1\) and \(\sigma_g^2 = \sigma^2 = 0.1, ~ \forall ~ g\)
  • Model 2: \(M = 4\)
  • Model : \(M = 4\), \(D_g \sim bern(0.5)\) and \(\sigma_g^2 = \sigma^2 \ = 0.1, ~ \forall ~ g\)
  • Model : \(D_g \sim bern(0.5)\) and \(\sigma_g^2 = \sigma^2, ~ \forall ~ g\), and \(\sigma^2\) is unknown (uniform prior)
  • Model : \(D_g \sim bern(0.5)\) and \(\sigma_g^2 \sim \text{half-}t(\dots), ~ \forall ~ g\)

Model 1

The only thing this model has to do is estimate the \(\beta_{g0}\) and \(\beta_{g1}\) parameters, since the variance is known and \(D_g = 1\) (i.e. no classification problem). M = 10

Model 2

M = 4

Model 3

Introduces the two-class component of the problem. M = 10

Model 4

Introduces the two-class component of the problem. M = 4

Model 5

Introduces variance estimation, i.e. estimation of \(\sigma^2\) instead of assuming the correct value

Model 6

Introduces estimation of variance parameters for \(\beta_{g0}\) and \(\beta_{g1}\), instead of assuming both are 1

Model 7

Adds paired random effects

Model 8

Switches to Poisson response